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ABSTRACT 


The vehicle classification and detecting its license plate are important tasks in 
intelligent security and transportation systems. The traditional methods of 
vehicle classification and detection are highly complex which provides coarse¬ 
grained results due to suffering from limited viewpoints. Because of the latest 
achievements of Deep Learning, it was successfully applied to image 
classification and detection of objects. This paper presents a method based on 
a convolutional neural network, which consists of two steps: vehicle 
classification and vehicle license plate recognition. Several typical neural 
network modules have been applied in training and testing the vehicle 
Classification and detection of license plate model, such as CNN (convolutional 
neural networks), TensorFlow, Tesseract-OCR. The proposed method can 
identify the vehicle type, number plate and other information accurately. This 
model provides security and log details regarding vehicles by using AI 
Surveillance. It guides the surveillance operators and assists human resources. 
With the help of the original (training) dataset and enriched (testing) dataset, 
the algorithm can obtain results with an average accuracy of about 97.32% in 
the classification and detection of vehicles. By increasing the amount of the 
data, the mean error and misclassification rate gradually decreases. So, this 
algorithm which is based on Deep Learning has good superiority and 
adaptability. When compared to the leading methods in the challenging Image 
datasets, our deep learning approach obtains highly competitive results. 
Finally, this paper proposes modern methods for the improvement of the 
algorithm and prospects the development direction of deep learning in the 
field of machine learning and artificial intelligence. 

KEYWORDS: Convolution neural network , Vehicle classification , Vehicle License 
plate Recognition 
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I. INTRODUCTION 

Vehicle classification and detection have promising 
importance in the future. Because, it can be utilized in many 
aspects such as to analyze the urban traffic statistics, 
Advanced Driver Assistance System, against vehicle theft, 
large parking lot management, against vehicle escape, 
improvement of the road information acquisition and safety 
management of highway, electronic toll collection (ETC), 
traffic investigation and so on. 

One of the most important applications for object 


recognition in the neural network is the image classification 
tasks. However, the vehicle classification and detecting its 
license plate still face a few great challenges because the 
number of vehicle classes are very large and that some 
attributions of the vehicle are too close to identify. So, the 
process of training the neural network requires a great 
number of parameters, which often accompanies by over¬ 
fitting problems. The recognition results can achieve a 
higher degree of accuracy in the training set, but a lower 
degree of accuracy in the test set. 



Fig 1: Classification of Vehicle 
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Although the training data set is modest in size, the neural 
network itself does not embody distinct advantages in 
evaluation systems of higher recognition accuracy and 
shorter training time than other models. The predominant 
motives for it are limited computing sources and lengthy 
processing time for even a small network. However, one of 
the keys to these issues is the feature extraction and 
classification of detection images for vehicle rearview. 
Because of Facing many practical problems, neural networks 
have been gradually integrated with some better methods 
emerging in the 21st century, such as Convolutional Neural 
Networks, ResNet Architecture, Tensor Flow. In this kind of 
model, aimed at different tasks, different systems are usually 
designed and respectively applied different manual design 
features. 



Fig 2: License Plate Recognition 


For example, the object recognition uses Feature extraction 
from CNN. License plate recognition uses Tesseract-OCR, and 
the pedestrian detection uses Histogram of Oriented 
Gradient (HOG) feature. Therefore, this paper aims at 
classifying and detecting the vehicles from different 
viewpoints under any circumstances. 

II. EXISTING SYSTEMS 

The traditional methods of vehicle classification and 
detection are mainly based on the following methods: 1) 
SIFT (Scale-invariant feature transform) feature matching 
and extraction; 2) the moving vehicle detection method of 
Gaussian mixture model; 3) the license plate classification 
method; 4) the monitoring video classification method of 
HOG (Histogram of Oriented Gradient) and SVM (Support 
Vector Machine). Based on these traditional methods, the 
neural network model gets the lower recognition reference 
to detect the objects. So, these methods give the inaccurate 
results in which accuracy is limited to 85% only. 

The above traditional algorithms have already obtained good 
results. Several shortcomings in the traditional methods 
have limited its realization: (i) the high similarity between 
models naturally influence the accuracy, (ii) some models 
containing only a few images, and (iii) the unified direction 
like front, side or back in picture causes the inaccurate 
classification, (iv) the methods usually require the images 
are captured from certain viewpoints, (v) the vehicle 
recognition granularity just stays in the stage of the vehicle 
model classification. In this paper we address these 
limitations and providing techniques that are practical for 
vehicle classification problem. According to existing 
methods, this paper further optimized the classification and 
detection algorithms. It applies the Deep Neural Networks 


(DNNs) to the classification and License plate detection of 
the vehicles and obtained a better recognition performance 
under different viewpoints or traffic conditions. 

III. LITERATURE SURVEY 

[1] Chen, Z., Ellis, T., and Velastin, S. A., "Vehicle Type 
Categorization" A comparison of classification schemes In 
Intelligent transportation systems (ITSC), 14th international 
IEEE conference. 

This Paper proposed the Vehicle detection and classification 
based on histogram of orientation gradients (HOG) approach 
which was presented by Zezhi Chen and Tim Ellis. This 
Classification model contains two parts: feature extraction 
and classifier selection. In their approach, measurement- 
based features (MBP) and the histogram of orientation 
gradients (HOG) features are used to classify the vehicles 
into four categories: car, van, bus, and motorcycle. 

For this classification model, they have used two classifiers. 
They are Random Forest (RF) and Support Vector Machines 
(SVM). Due to the multiple classification problems, they 
decided to use one-vs-all strategy. With this approach, the 
best 3D model can be found that match with the original 
vehicle type. The model they have presented is a novel 
vehicle type classification, which provides an accurate and 
reliable performance. However, the features among the four 
vehicle types are very different. Sometimes 3D models of two 
vehicles such as hatchback and SUV may be similar. In this 
case, this system may be hard to detect the vehicle type. 

[2] Chang, S.-L., Chen, L.-S., Chung, Y.-C., and Chen, S.-W., 
"Automatic license plate recognition" IEEE transactions on 
Intelligent Transportation Systems. 

This paper proposed a license plate image technique 
consisting of two main models: a license plate locating model 
and a license number identification module. Specifically, the 
license plates extracted from the first model are examined in 
the identification model to reduce the error rate. They used 
color edge detection to compute edge maps. 

But this is limited to only four types of edges. By using the 
unique formulas, the model can transform RGB space into 
HSI space that denote red, green, blue colors as hue, 
saturation, intensity parameter values of an image pixels, 
respectively. The identification module consists of two main 
stages, preprocessing and recognition. After this process, 
segmentation and recognition will be invoked sequentially. 
However, this identification model takes more time to 
recognize the characters and it is a complex process which 
needs to be modified. This model can detect the License 
plate, if and only if that license plate & characters are in 
specified color edges. This type of model can be useful in a 
particular region or place. 

[3] Farhat, A., Al-Zawqari, A., Al-Qahtani, A., Hommos, 0., 
Bensaali, F., Amira, A., and Zhai, X., "Tesseract-OCR Based 
Feature Extraction and Template Matching Algorithms For 
Number Plate Recognition" In Industrial Informatics and 
Computer Systems (CIICS), International Conference on 
IEEE. 

A modified template matching Correlation algorithm was 
proposed by Ali Farhat et al. They have developed four 
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algorithms for Numeric Automatic License Plate Recognition 
(ALPR) systems: Vector crossing, zoning and combined 
zoning-vector and template matching correlation. First three 
algorithms are based on feature extraction techniques and 
the last one is application of correlation technique. By using 
vector crossing algorithm, they distinguished the ten 
characters (0-9) except the characters "2", "3" and “5". Since 
these characters have the same number of the vectors. By 
using Zoning method, densities of the image in each zone are 
derived by the algorithm to determine the characters. 

The third approach is the combination of the previous two 
methods. In that, density of the image is calculated. However, 
these three algorithms cannot determine a noise in the 
image. Hence, they developed a system based on template 
matching correlation as a fourth approach. This correlation 
indicates the linear relationship between each character 
with all other characters. However, the limitations of this 
approach are that, they can recognize only 10 numbers 
instead of the alphabets. If more templates are added to the 
system, the success rate may decline since some characters 
share the same density and vector. 

[4] Casey, R. G. and, E., "A survey of methods and strategies 
in character segmentation", IEEE transactions on pattern 
evaluation and machine intelligence. 

Lecolinet et al proposed a new algorithm for license plate 
identification. For faster detection of license plate regions, 
they developed a novel method called Sliding Concentric 
Windows (SCW), which can describe the irregularity in the 
image based on image statistics. The SCW segmentation 
algorithm involves following steps, (i) Two concentric 
windows A and B were created for the upper left corner pixel 
of the image, (ii) Mean values are obtained from both 
windows. In this approach, the Rol (Region of Interest) will 
be detected if the ratio of the statistical measurements (B to 
A) in the couple of windows reaches a threshold value. 

Image masking, Binarization and connected components 
analysis (CCA) functions are employed in sequence. As a 
result, license plate region is detected. In license plate 
processing stage, they created an approach to segment the 
characters. The probabilistic neural network (PNN) is 
invoked to recognize the characters. As a result, possible 
character is determined in the output layer. 

However, the proposed system relies highly on the lighting 
condition and the physical appearance of the plates. In some 
cases, the model cannot detect the plate due to the 
illumination condition or the quality of the plates. 

IV. PROPOSED SYSTEM 

This paper proposed an Advanced vehicle classification and 
detection method based on deep neural networks. In this 
system, vehicle classification task is accomplished by 
Convolutional Neural Networks. It extracts the vehicle region 
from its background area by pictures, in order to test the 
generalization ability about the domestic vehicle types of 
vehicle dataset. So, the vehicle region is segmented to 
remove background noise to improve the recognition 
accuracy. In this approach, the convolutional neural network 
is used to complete feature extraction and classification 
training of labeled image datasets of many vehicle types such 


as bus, car, bike, bicycle. In this way, the task of identifying 
vehicle types from different angles can be completed. 



Fig 3: Input Image to the Model 


Besides, this paper also focuses on automatic detection of the 
vehicle License plates which can be accomplished by 
Tesseract-OCR (optical character recognition). It is used to 
recognize the characters and provides the text from the 
given image. So, the extracted License Plate is given to the 
tesseract-OCR which in return provides the characters of 
number plate for license plate recognition. 



Fig 4: Output Image from Trained Model 


Finally, the Proposed model is evaluated based on selection 
accuracy, recognition accuracy and real-time performance in 
the vehicle classification and detection process. While 
training the model for the detection of vehicle type and its 
number plate, the dataset must be split into train data and 
test data. The train and test data are statistically known as 
positive overlap range and negative overlap range 
respectively. 


When we dataset the parameter "Negative Overlap Range" 
(where the overlap ratio between the candidate and the 
image labeled area is used as the interval for negative 
training samples) to [0, 0.3], and the parameter "Positive 
Overlap Range" (the overlap ratio between the candidate 
and the image labeled area as the interval of the positive 
training sample) to [0.7, 1.0], the accuracy of proposed 
vehicle detection model will be the highest. 


V. SYSTEM ARCHITECTURE 

Block Diagram of the Proposed System is represented as 
shown below in figure 5. This Proposed block diagram is 
categorized into three parts. They are (i) System Framework, 
(ii) Software Requirements, (iii) Methodology 
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Fig 5: Block Diagram of Proposed System 


A. SYSTEM FRAMEWORK 

1. Vehicle Detection and Classification Based on 
Convolutional Neural Networks: 

1.1. Vehicle Detection: 

In this system, images of the training dataset are given as 
input and the foreground area of the target vehicle is 
detected by the convolutional neural network (CNN). But 
there are some issues to build a deep learning model by 
using CNN. The first problem while training the deeper 
networks is, accuracy should be increased with an increase 
in depth of the network as long as over-fitting is taken care 
of. However, the problem with increased depth is that the 
signal required to change the weights, which arises from the 
end of the network by comparing ground-truth and 
prediction becomes very small at the earlier layers, as a 
result of increased depth. 

It essentially means that earlier layers are almost negligible 
to learn. This is called a vanishing gradient. The other 
problem while training the deeper networks is, executing the 
optimization on huge parameter space and therefore naively 
adding the layers which results in higher training error. This 
is called a degradation problem. In order to overcome these 
issues, Convolutional Neural Network uses ResNet 
Architecture to train the deeper networks. 

ResNet architecture makes use of shortcut connections to 
solve the vanishing gradient and degradation problems. The 
basic building block of ResNet is a Residual block which is 
repeated throughout the network. The ResNet structure as 
shown in Figure 6. 



Fig 6: A Residual Block of Deep Residual Network 


an identity function and the shortcut connection is called 
Identity connection. The identical mapping is learned by 
zeroing out the weights in the intermediate layer during 
training, since it's easier to zero out the weights than push 
them to one. For the case when the dimensions of F(x) differ 
from x, Projection connection is implemented rather than 
Identity connection. The function G(x) changes the 
dimensions of input x to that of output F(x). 



x 

Fig 7: Projection connection of Residual Network 


ResNet Architecture uses Rectified Linear Unit (ReLu) as an 
activation function of neural networks. This rectified Linear 
activation function is a piecewise linear function that gives 
the output as its input itself, if the input is positive. 
Otherwise, it gives output as zero if the input is negative. 
ReLu activation function reduces the large number of 
computations. It has become the default activation function 
for many types of neural networks because a model that uses 
ReLu is easier to train and often achieves better 
performance. The ReLu activation function is given as shown 
below. 


F (x) = 



if x > 0 
if x < 0 


► for all x eR 


By using ReLU as an excitation function instead of the 
traditional function, Logistics makes it easier to forward 
propagation and calculate the reverse gradient by using 
partial derivative. It also helps avoid complex algorithms 
such as exponent and division. 

The hidden layer neurons with output less than 0 are 
discarded to increase the sparsity and reduces the 
overfitting effect. The ResNet network structure is adjusted 
while testing the data and the extracted vehicle area is 
outputted as shown in Figure 8. 


Instead of learning the mapping from x -> F(x), the network 
learns the mapping from x —> F(x)+G(x). When the dimension 
of the input x and output F (x) is same, the function G(x) = x is 
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Fig 8: Vehicle Detection 


1.2. Vehicle Classification: 

The foreground vehicle area obtained by vehicle detection 
algorithm is introduced into another convolutional neural 
network for vehicle recognition. Convolutional neural 
network uses a certain number of convolutional kernels to 
slide the input image with a certain step size to extract 
features in the convolution layer. Each convolution kernel 
focuses on different features, so the convolution obtains 
different feature maps. 



ju.Lukl 


Fig 9: CNN Architecture 

These obtained feature maps are passed into the lower 
convolution layer, and then the lower convolution layer uses 
a certain number of convolution kernels to extract the 
features of upper feature map, this operation will be 
repeated. The weighting parameter will be got after the 
features are processed for several times. Then the weighting 
parameter will be connected to the full connection layer, the 
full connection layer plays a role of classifier in the entire 
convolution neural network, namely a model to classify the 
extracted features. 

By using the method of Deep Learning, the features ignored 
by the traditional methods can be extracted, the recognition 
accuracy can be improved obviously, and different 
convolution kernels are used to obtain convolution feature 
diagrams, as shown in Fig 10. This ResNet architecture is 
similar to the VGGNet which consists mostly of 3X3 filters or 
kernels. 



Fig 10: Feature Map of Convolution Layer 


From the VGGNet, shortcut connection is inserted to form a 
residual network. In order to overcome the issues such as 
vanishing gradient and degradation problems of the deep 
learning model which occurs during training the model. 

2. TENSORFLOW: 

TensorFlow is an open source library which uses data flow 
graphs to build the models. This allows developers to create 
large-scale neural networks with many layers. TensorFlow is 
a neural network module which is mainly employed for 
Classification, Perception, Understanding, Discovering, 
Prediction and Creation. Tensors are the multidimensional 
arrays, an extension of 2-dimensional tables to data with a 
higher dimension. There are many features regarding 
TensorFlow which makes it appropriate for Deep Learning. 



■Jw fr.'-U 


Fig 11: TensorFlow Architecture 


TensorFlow architecture works in three parts. They are (i) 
Preprocessing the data, (ii) Build the model, (in) To train and 
test the model. Initially, the raw data is scaled by 
normalizing the input and output features of a model. This 
preprocessed data splits into train and test dataset. This 
Building model is trained using train dataset. By comparing 
the prediction accuracy of test dataset with the ground-truth, 
loss function of the model can be analyzed and tuned with 
hyperparameters. TensorFlow is used for fast numerical 
computations of neural networks. It is a foundation library 
that can be used to provide Deep Learning models directly or 
perhaps by using wrapper libraries that simplify the process 
built on top of TensorFlow. 


3. OpenCV: 

OpenCV is also known as Open Source Computer vision. It 
is a cross-platform library of programming functions 
which mainly aims to develop the real-time computer 
vision applications. OpenCV was developed by Intel 
organization. This mainly focuses on image processing, 
video capture and analysis which include features such as 
face detection and object detection. It supports some 
models from deep learning frameworks such as tensor-flow, 
Tesseract-OCR, PyTorch modules to build an effective model. 


By integrating the OpenCV with the Tesseract-OCR, it can 
extract the characters even from colored images with higher 
accuracy. 


B. SOFTWARE REQUIREMENTS 
1. TESSERACT-OCR: 

The Character Recognition popularly referred as Optical 
Character Recognition (OCR). Tesseract-OCR is the 
software process of converting typed, handwritten, or 
printed text to machine-encoded text that we can access 
and manipulate via a string variable. The OCR has been 
one of the challenging and popular fields of research in 
pattern recognition. This can be used as one of the 
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classifier in the image processing step since the most 
crucial step in OCR is identifying characters as one of 36 
symbols (A-Z and 0-9). Therefore, for classification, we 
need to extract visual features from individual character 
image. Then, character can be classified with these 
features using machine learning. The most effective way 
for now is OCR-tesseract, which is originally developed by 
Hewlett Packard in the 1980s. It is one of the most 
accurate open source OCR engines currently available. 



Fig 12: Flowchart of Tesseract-OCR Process 


As other traditional procedures, this processing follow a 
step by step approach. As the most important part, 
recognition consists of two stages, which are adaptive 
classification and repeating recognition, respectively. In 
the first stage, each character is recognized sequentially. 
Additionally, the symbol that is classified is stored by the 
adaptive classifier as the training data. Then, the adaptive 
classifier will learn the information that classified 
characters provide, from which the characters that are 
not recognized in first step will be classified again. This 
two-stage process make this method accurate and 
efficient. 

2. LabellMG: 

Labelimg is a graphical image annotation tool. It is designed 
in Python and uses Qt for its graphical interface. Annotations 
are saved as XML files in PASCAL VOC format which is 
employed by ImageNet. Image Annotation could be 
the process of building datasets for computer vision models. 
This helps machines to learn, how to automatically assign 
metadata into a digital image using captioning or keywords. 
This Labelimg tool is used to train our customized model. 

This technique is used for image retrieval systems to 
organize and easily locate particular images from a database. 
Image labeling gives the insight into the content of images. 
When you use the API, you get a list of the entities that were 
recognized: people, things, places, activities, and so on. 
Each label found comes with a score that indicates the 
confidence or probability or accuracy of the ML model 
according to its relevance. 


3. ANACONDA NAVIGATOR: 

Anaconda Navigator is used to launch applications and to 
manage conda packages, virtual environments, and 
programs without the use of command line commands. In 
order to get the Navigator, download the Navigator Cheat 
Sheet and install Anaconda. It is a free open-source software 
which is associated with Python and R programming 
languages for scientific computing such as for data science, 
machine learning applications, large scale data processing, 
predictive analytics and so on. It aims to simplify package 
management and deployment. 

Jupyter Notebook is a one of the IDE which have to be 
launched from the Anaconda Navigator to develop, execute, 
debug the algorithms which performs Artificial intelligence 
tasks. It provides you with an easy-to-use, interactive data 
science environment across many programming languages 
that doesn't only work as an IDE, but also as a presentation 
or education tool. It's perfect for those who are just starting 
out with data science. Because, it helps to debug the errors 
by specifying the type of error that was obtained. 

C. METHODOLOGY 

1. DATA COLLECTION & DATA PREPARATION: 

In this Project, dataset is prepared by the collection of 
vehicle images from Open source, CCTV cameras and 
internet. To ensure the representativeness and the integrity 
of data, images are taken from different viewpoints under 
different traffic conditions. Most of the images in this dataset 
are rear views of the vehicle. Besides, the image dataset also 
covers most types of vehicles and noise images. 

This involves collection of data by recording using a camera 
with auto iris function which keeps the average illumination 
of the view constant. It also uses i-LIDS dataset, then the 
input data is processed into a set of features before 
becoming suitable inputs for per frame vehicle detection and 
classification using 3D models. 

2. EDGE DETECTION: 

An edge based multi-stage detection is the main primary 
function to detect the vehicle and its license plate edges at 
their localization. In this paper, license plate edges are 
detected from its original image by edge detection. 
Localization of plate is undoubtedly a challenging task 
since there are significant variations in plate size, color, 
lighting condition and spatial orientations of license plate 
in images. So, three steps are employed at the 
preprocessing stage. They are (i) Gray Scale Conversion, 
(ii) Median Filtering, (iii) Contrast Enhancement. 

2.1. Gray Scale Conversion: 

By using following formula, the 24-bit color image can be 
converted into 8-bit gray image. 

Gray = 0.59xR + 0.30xG + O.llxB 

2.2. Median Filtering: 

As one of non-linear filter, it can calculate the median of 
the gray values of a pixel's neighbors. In this stage, they 
use 3x3 masks to get eight surrounding neighbors' gray 
value and replace the pixel value with the median value. 
As a result, the function could remove salt-and-peeper 
noise from the image. 
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2.3. Contrast Enhancement: 

Histogram equalization technique is invoked to enhance 
contrast of the images. In the procedure of the conversion, 
the total number of pixels in the image is N and the 
number of pixels with the gray level k is nk. Then the 
stretched gray level Sk is calculated by the following 
formula. 

S k = EjUf * 255 


3. FUNCTIONAL PROCESS: 


This paper proposed an Advanced vehicle classification and 
detection method based on the deep learning approach. In 
this system, the surveillance cameras are used for observing 
the vehicles at any specific region or area. These surveillance 
cameras are often connected to the Personal Computer (PC) 
or any recording device to provide those vehicle images or 
videos which are passing through that specific gateway or 
area. This vehicle data is taken as input to the Personal 
Computer (PC). By using the i-Lids, real time camera and 
open source data a dataset is created. With the help of this 
dataset, the Deep Learning model is trained up- to a specified 
epoch. With the Increase of number of epochs, the loss 
function of the model is reduced. Finally, the trained model is 
tested with the new vehicle data to check whether the model 
is working effectively in vehicle classification and detection 
of their number plates. 
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Fig 13: Functional Process of Proposed Scheme 


In this system, Vehicle classification method is based on a 
convolutional neural networks and automatic license plate 
detection of vehicles method is based on Tesseract-ocr. This 
Deep neural network model will detect the vehicle from the 
given image or video by using frame extraction. It also 
provides the classified type of vehicle whether it is bus or 
bike or bicycle or any other type. 


The model will detect the vehicle number plate if and only if 
there is number plate for that detected vehicle. The detected 
vehicle number plate is extracted as an image from the given 
data and simultaneously given to the Tesseract-OCR which is 
integrated with the OpenCV to recognize the characters 
present in that number plate. The model will be instructed to 
detect and classify the vehicles, only when their detection 
probability is more than 0.90. This Probability approach will 
be useful to get the accurate vehicle info or effective results 
of the model. A log is created in the PC, to note the details of 
the vehicles. 


VI. FLOW CHART 



Fig 14: Flow Chart of Proposed System 


The given flow chart of proposed system explains the 
procedure and sequence of Framework as shown above in 
figure 14. 

VII. EVALUATION & DISCUSSION 

For evaluating the proposed or implemented model, we 
must compare the performance of the proposed system to all 
other existing systems. After collecting the relevant training 
and testing experiment information, we analyzed and 
compared the accuracy rate of the different network 
structures such as AlexNet, Vggnetl6, Vggnetl9, Googlenet, 
Resnet50 and ResnetlOl. Currently, proposed system uses 
ResNetlOl architecture to train and test the deep learning 
Algorithm for vehicle classification and detection. The 
accuracy for different network architectures shown below. 


CR (%) = 


CN (c urrart) 

TN( CTjV (c orr&et } +■ EJf (■e-rt'Df'J] 


In the equation, CN means for the number of correctly 
classified images and EN means for the number of 
incorrectly classified images indicates the correct rate. To 
compare the experimental results of various network results, 
the graphical representations of them are given below. 


This Info includes type of vehicle, its number plate and other 
info like time in & time out are tabulated accurately in the 
log. Every vehicle info is appended to its log. Finally, this 
model provides the accurate info of log regarding the 
vehicles that are passed through this Artificial Intelligence 
Surveillance. 
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Fig 15: (a) loss curves of curves; (b) curves of accuracy 


The dataset with 3500 pictures is collected and 
preprocessed with auto-iris function which is used to 
evaluate the trained models. We drew the loss curve and 
accuracy curve of every classification and recognition 
network structure, which are helpful to understand the 
training process of every model. From the comparison of the 
curves, it is clear that the Resnet-101 can get the fastest 
convergence and higher accuracy as shown in Fig 15. 


Architecture 

Accuracy 

Year 

Alex Net 

82.6 

2012 

Inception-VI 

88.4 

2013 

VGG 

90.8 

2014 

ResNet-50 

92.3 

2016 

ResNet-101 

95.2 

2018 


Table 1: Comparision of Network Structures 


We apply the proposed model which is trained by the image 
dataset into the real traffic road video or images collected by 
ourselves. The collection of real traffic vehicle pictures is 
also tested. We observed that for the recognition of the real 
traffic road image, the training image dataset obtains the 
higher accuracy. 


The experimental results obtained from the training and 
testing dataset is shown in the below table. The figure 15 
shows that, Resnet Network structure gives the higher 
accuracy for object detection and classification than the 
other network structures. By analyzing the testing 
performance with different number of layers in ResNet 
structure, the best performing one is the cascading of the 
ResNetlOl network structure which achieves 95.27%. 


VIII. RESULTS 

In most of the cases, the experimental results of proposed 
system have obtained with higher accuracy. The proposed 
method has integrated feature extraction, object frame 
boundary generation, linear regression & classification and 
License plate recognition to provide a customized efficient 
model. By using this model, we can retrieve the vehicle info 
accurately. Even if there is background noise in the license 
plate, it will not affect the performance of the recognition. So, 
this system can recognize all the characters on the license 
plate successfully. Thus, the comprehensive performance has 
been greatly improved. When compared to existing models, 
the current deep learning vehicle classification and detection 
neural network model has the rate of accuracy as very high. 
Therefore, the proposed model gives standard results. 

IX. CONCLUSION 

Based on CNN, this paper proposed the vehicle type 
classification and license plate recognition in urban traffic 
video surveillance. This Deep Learning model can do both 
Classification and Detection of vehicles simultaneously. This 
reduces the complexity of the processing which helps to 
increase the performance of the system to train and test the 
model. With the increase of training dataset, the modification 
of parameters and replacement of the model; the proposed 
method becomes effective and validated by the relevant 
experiments. The experimental results show that the vehicle 
recognition rate is improved. In a comparative analysis, this 
vehicle classification & detection framework can get high 
accuracy with success rates which has been proposed to 
expose in a better performance than existing frameworks. 

X. FUTURE WORK 

In the future, we will implement this proposed model as a 
real-time hardware application with a deep learning 
Framework which further improves the accuracy and 
robustness for the traffic vehicle classification and detection. 
There are still mountains of work about deep learning to be 
studied. For example, it needs the other efficient and theory 
based deep learning model algorithms. It explores the new 
feature extraction models which are also worthy to do 
further research. Besides studying efficient parallel training 
algorithms, they need to be investigated. Regarding the 
applications of extending deep learning and how to make 
rational use of deep learning in enhancing the performance 
of traditional algorithms is still the focus of a variety of fields. 
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